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K sis, the driver. In the long term, AVCS wi.1 provide fully automated vehide/highway systems 
replacing the human driver altogether. . 

The use of vision-based perception a P rimaI > 

to odter appmaches. 

This paper describes the testbed 

SS‘^Ss“ ^Srit^Uon^averse. and autonomous dnvtng. 

2 TESTBED AND SUPPORT VEHICLES ^ ^ Tooele Army Depot as part 

S£*l d^S: Srsot C .o^rrrd='d gauges indicating speed. RPM, and 
temperature. 



Figure 1. Testbed vehicle followed by support van 
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following on public roadways. During la g image information is processed 
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mforma&° n . The world modeling modules estimate the state of the external world and make 
KJfrim^ n ni and CVaIUan0n ? baS ^ ed 0n •* estim ates. The task decomposition modules implement 

and m °5, lt ?u ng functions * 1,16 roles of these submodules are P further 
t i 81 * ^ refere . ncc model has not fully implemented but has served as a guide 
as various control nodes were completed and as the vehicle’s capaK 
increased from teleoperation to autonomous driving. ^ ' 



The highest level of control for an individual vehicle, the Task level module, executes mission tasks 

L^cnKc > T b0llC Sucb as - : myc t0 exit 1 1 on 1-270. A vehicle may be SpeSS 

T^It i^w SySt K^ S ’ SUCh “ navi S au ° n - perception, and mission modules, which are directed by the 
Task level to achieve certain phases of the task. y 

f ° r ^ Vu' Al J ny Research Lab at Demo I used the lower two levels. Prim and 
Servo, of the mobility pan of the reference model architecture to perform the mission elements The 
servo level mobility controller drives motors for steering, brake, throttle SmissioTetc and 
monitors the dashboard gauges. Vehicle navigation sensor data (position, velocity and acceleration! 
is processed and used to update the WM in the lowest level of the navigation subsystem. This data 
is used for steering and speed control of the vehicle during retro-traverse. 

foUo^naifsf' SyS !? m i™*, «« ssai >f0f implementing the IVHS autonomous road 

following [4, 5]. The lower two levels, Pnm and Servo, on the perception side of the generic 
«hcle control system were developed. See Figure 2. The vision Option s«tem u“s a mS 
of the lane edges to assist in the prediction and tracking of the lane marker on the road The 

to ° f ±C CCntCr ° f lanC “* med 10 Steer thc vehicle * in a s^ar fashSn 


Car f( ? lowing a J} d coIlis i° n avoidance requires the implementation of the next 
higher level of the conn-ol system, Emove. In this case the control system uses the visual surface 
features of the rear of the lead vehicle for lateral/longitudinal control in order to perfonn platooning 
[ ]. Eventually, the performance of higher level tasks such as obstacle recognition/avoidance and 
the ichitSture^ 0 rcqmre fuTther cxtensions to Emove and implementation of the Task levels of 
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4. APPLICATIONS 

Akhoifg^ die ultimate goal for robotic vehicles is a fully autonomous system, coH^^hnoJo^ has 
mlSncdf^ enough to realize this goal. Some form of operator intervention is needed, at leas 
naTt of the time For IVHS needs, the driver resumes control when the automatic system can n 
Cta. to siring the vehicle is unmanned and operator control requires some form of 

teleoperation. 

AW. vchiclescommun^catt toa A 

vehicles at one wne. Tank Automotive Command. Each system allows the operator to 

SSSSKSte — are issued usi/g a much semen dfTplay. A 

graphic display presents vehicle status to the operator. 

feedback, graphic overlays, and delay compensation. 

f~rihack of the steering wheel provides the operator a feel for road conditions as well as sense 
^"d veMcle spcfd. Unfortunately, doling a high speed loop dace 

incised so the 8 wheel is not allowed to turn past a limit which is a function of speed. 

fr^sLT^hifSy m represent the position of where the vehicle will travel at the given 
steeringposMom The projectedrehicle position represented i n th e v ideo assumes a flat ground 
plane andmoves further ahead of the vehicle as forward speed increases. 

Knallv we are investigating controller delay compensation. During teleoperation, several steps 
Finally, we are invesngaimg w . J ■ * it ■ transmitted to the control station, the 

determined. 
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[12]. The operator specifies the desired velocity and selects an automatic turnaround manuver. The 
Modular Azimuth Position System (MAPS), an inertial navigation unit, is used to sense vehicle 
position and. orientation. MAPS uses ring-laser gyros and accelerometers to determine vehicle 
motion. An interface board (called the Navigation Interface Unit) and software to integrate vehicle 
odometry with MAPS data was developed by Alliant Tech and used during Demo I. Details of the 
navigation portion of the driving package are in [3]. 

Autonomous Driving 

There are two low level functions required to drive a vehicle down the road, stay on the road and do 
not hit anything. NIST has been developing a vision based perception system to perform these 
functions. 

The controller tracks the lane markers commonly painted on roadways and steers the vehicle along 
the center of the lane in the following steps. First, edges are extracted from the video image within a 
window of interest. Edges occur where the brightness of the image changes, such as where the 
unage changes from a gray road to a white stripe. Then, quadratic curves that represent each of the 
two lane boundaries as they appears in the video image are updated. The system computes the 
coefficients of the curves using a recursive least square fit with exponential decay. The steering 
wheel angle that steers the vehicle along the center of the perceived lane is calculated using the pure 
pursuit method used for retro-traverse. Finally, navigation sensors compensate for the computation 
and transmission delay by adjusting the steering goal in accordance to the motion of the vehicle 
during the delay. More details of the vision processing and control algorithms can be found in [4,5]. 

Figure 3 shows the various scenes obtained when applying a window of interest to the road scene. 
The lateral position of the window of interest shifts in order to keep it centered on the road and its 
shape changes as a function of the predicted road curvature. 



Figure 3. Road Scene, Window of Interest, Masked Road Scene. 

The Montgomery County DOT permitted NIST to test the instrumented vehicle on a public highway. 
During these tests, autonomous driving was maintained over several kilometers (gaps in the lane 
markings at intersections prevented test runs of longer distances) and at speeds up to 90 Km/h. The 
vehicle has also been driven on various tests courses under weather conditions ranging from ideal to 
heavy rain, and under various outdoor lighting conditions including night time with headlights on. 

Besides following the road, an autonomous vehicle must track and avoid obstacles and other 
vehicles. In addition, if the system can track another vehicle, it can follow that vehicle, forming a 
platoon. Platooning is envisioned by the military to reduce manpower requirements. In the IVHS 
version, vehicles would platoon at two meter spacings, in order to increase traffic throughput. 
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An approach to vision-based car following was developed that tracks the back of a lead vehicle or a 
target mounted on the back of the vehicle [9]. Since orientation is approximately constant during car 
following, the algorithm estimates only the relative translation of the lead vehicle. TTie system was 
tested using a video recording taken while the testbed vehicle was manually driven behind the lead 
vehicle. The system demonstrated tracking for vehicle separations of up to 15 meters. 

NIST's roles are to evaluate component technology for autonomous vehicles and to work with 
industry and academia to advance the state-of-the-art. To perform such a task, an architecture has 
been developed that will allow incremental development of autonomous capabilities iin a modular 
fashion. The low levels of the control system have been implemented to support the DOD near term 
robotic tech base. That system was demonstrated at the 1992 Demo I. The control system was 
systematically extended to incorporate higher levels of autonomous capabilities to support further 
evaluations and developments in conjunction with the DOD tech base and DOT IVHS programs. 
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